The explosive growth of the Internet has transformed data centers into large industrial-scale computer facilities with extraordinarily high energy demands. From Google and Facebook to banking, cloud computing and supercomputing, an average data center already uses as much electricity as a medium-size town. In Silicon Valley, data centers are also listed as the top air polluters from backup diesel exhausts. Already by 2012, the energy costs for a data center were estimated to exceed the cost of the original capital investment over its useful life. The carbon footprint of data centers is expected to exceed that of the airline industry by 2020. For 2011, the Facebook carbon footprint was ˜285,000 metric tons of CO2 equivalent. For 2010, that of Google was five times higher—1,500,000 tons. Energy considerations are forcing the construction of new data centers in areas where the climate helps cooling and electricity is cheaper. A recent Facebook 120 MW data center was built just south of the Arctic Circle in Sweden, close to a hydropower station producing twice as much electricity as the Hoover Dam in Nevada. Besides just high energy costs and adverse environmental impact, there is a compelling technical reason to improve the energy efficiency of computing technologies. The development of the next generations of high-end computers (e.g., exascale supercomputers and beyond, where exa=1018) will not be possible unless a significant improvement in energy efficiency is achieved over the technology available today. See, for example, R. Service, “What'll It Take to Go Exascale”, Science Magazine, vol. 335, p. 394, Jan. 27, 2012, expressly incorporated herein by reference. For a computer rated at 1 ExaFLOPS (109 Giga FLoating-point OPerations per Second), this requires >50 GigaFLOPS/W. As of November 2012, the fastest supercomputer Titan (Cray XK7) had ˜2 GigaFLOPS/W (˜20 PetaFLOPS at ˜10 MW). The power dissipation target for a future exascale supercomputer is very stringent—no more than 20 MW, which is just two times larger than that of Titan with ˜ 1/50 ExaFLOPS.
The heart of the problem is in the relatively low energy efficiency of current computer circuit technologies that consume too much power for computing, storing and moving data between processors and memories. Despite the fact that Moore's law continues to enable even more transistors per chip, Dennard scaling (the simultaneous reduction of CMOS threshold and bias voltages commensurate with device size reduction) ended a few years ago. Now every new CMOS process generation has higher power density, and peak power requirements are increasing at a rate far exceeding the ability to remove heat. This is the reason that energy efficiency rather than switching speed or circuit area has now become the dominant metric in computing performance, from hand-held portable devices to high-end, large-scale supercomputers.
Conventional approaches are unlikely to yield sufficient reduction in power density. In contrast, superconducting single-flux quantum (SFQ) circuits, by virtue of their inherent low power dissipation, high speed, and lossless interconnect, present an excellent opportunity to dramatically increase the energy efficiency of high-end computing applications. See, for example, D. S. Holmes, et al., “Energy-Efficient Superconducting Computing—Power Budgets and Requirements”, IEEE Transactions on Applied Superconductivity, vol. 23, no. 3, 1701610 (June 2013), expressly incorporated herein by reference. This should dramatically enhance the energy-efficiency of data centers and enable new generations of supercomputers.
Ever since the late 1960s, superconducting Josephson junction integrated circuits have been considered as possible candidates for high-speed, low-power computing. See, for example, W. Anacker, “Josephson Computer Technology: An IBM Research Project”, IBM Journal of Research and Development, vol. 24, no. 2, p. 107 (March 1980), expressly incorporated herein by reference. See also U.S. Pat. Nos. 5,365,476; 4,509,146; 4,360,898; 4,633,439; 5,126,598; 5,388,068; all expressly incorporated herein by reference. This technology produced circuits with very low power densities and clock rates of several GHz, fabricated using a robust integrated circuit process based on niobium Josephson junctions, typically operating at a temperature near 4 K. For more on the niobium IC process, see e.g., U.S. Pat. Nos. 4,430,662; 7,615,385; 8,383,426; 4,499,199; 4,589,161; 7,060,508; 7,105,853; 8,301,214; 8,437,818; 2011/0089405; 5,055,158; all expressly incorporated herein by reference. However, these earlier circuits were superseded by another much faster logic family also based on Josephson junctions, superconducting rapid single-flux quantum (RSFQ) logic, invented in the mid-1980s, which promised digital circuits with clock rates up to 100 GHz. See K. K. Likharev and V. K. Semenov, “RSFQ Logic/Memory Family: A New Josephson Junction Technology for Sub-Terahertz-Clock Frequency Digital Systems”, IEEE Transactions on Applied Superconductivity, vol. 1, no. 1, p. 3 (March 1991), expressly incorporated herein by reference. This enabled the development of ultrafast digital signal processing circuits by the mid-2000s, and today, cryogenic RSFQ Digital-RF receivers operating with 30 GHz clock frequency are available for wide-bandwidth satellite communications and signal intelligence applications. See, for example, O. Mukhanov, et al., “Superconductor Digital-RF Receiver Systems”, IEICE Transactions on Electronics, vol. E91-C, p. 306 (2008), expressly incorporated herein by reference. See also U.S. Pat. Nos. 8,462,889; 8,260,143; 8,260,144; 8,260,145; 8,521,117; 8,055,235; 8,521,117; 8,301,104; 8,514,986; 7,876,869; 8,045,660; 8,130,880; 8,514,986; 7,280,623; 8,249,540; 8,401,509; 7,701,286; 7,362,125; 7,991,013; 8,498,491; all expressly incorporated herein by reference. Furthermore, various prototypes of high-speed processors, data and signal processing modules have also been demonstrated. See, for example, A. Fujimaki et al., “Bit-serial single flux quantum microprocessor CORE, IEICE Transactions on Electronics, vol. E91-C, p. 342 (2008); M. Dorojevets, et al., Data-flow microarchitecture for wide datapath RSFQ processors”, IEEE Transactions on Applied Superconductivity, vol. 21, no. 3, p. 787 (June 2011); M. Dorojevets, et al., “8-Bit Asynchronous Sparse-Tree Superconductor RSFQ Arithmetic-Logic Unit With a Rich Set of Operations”, IEEE Trans. Appl. Supercond., vol. 23, no. 3, 1700104 (June 2013); all expressly incorporated herein by reference. See also U.S. Pat. Nos. 7,376,691; 7,440,490; 6,917,537; 6,865,639; 7,443,719; 7,903,456; 6,960,929; 7,459,927; also WO2002/069498; all expressly incorporated herein by reference.
RSFQ logic is based on exploiting single quanta of magnetic flux to encode clock and data, corresponding to a fast voltage pulse is generated with quantized area∫V(t)dt=Φ0=h/2e=2.06×10−15Wb=2.06 mV-ps,
known as a single flux quantum or SFQ. For a typical Josephson junction, such a pulse is created with pulse height ˜1 mV and pulsewidth ˜2 ps. The energy consumed during this switching event is of the order of IC×Φ0˜10−19 J assuming IC˜0.1 mA (chosen to exceed thermal noise at 4 K). Therefore, the gate switching energy is directly related to thermal energy rather than device dimensions as in CMOS. The picosecond quantized SFQ voltage pulses were proven to propagate ballistically on-chip and between chips via superconducting microstrip lines (with low loss and dispersion) without the need for amplification, and with speeds close to the speed of light. This is the key advantage of superconducting technology over CMOS, in which the data movement energy is proportional to the length of interconnect and currently represents the dominant share of the consumed energy.
Until recently, the inherently low switching power of conventional RSFQ logic was overwhelmed by the static power dissipation in the network of bias resistors used to distribute the required amounts of DC bias current for RSFQ gates. This overhead power was dissipated all the time regardless of circuit operation status. Recent efforts have resulted in significant reduction and even complete elimination of the static power dissipation in SFQ circuits. See, e.g., O. Mukhanov, “Energy-efficient single flux quantum technology”, IEEE Trans. Appl. Supercond., vol. 21, p. 760 (2011); Q. Herr, et al., “Ultra-low-power superconductor logic”, Journal of Applied Physics, vol. 109, 103903 (2011); M. Tanaka, et al., “Low-energy-consumption RSFQ circuits driven by low voltages”, IEEE Trans. Appl. Supercond., vol. 23, 1701104 (June 2013), all expressly incorporated herein by reference. See also U.S. Pat. Nos. 8,571,614; 7,724,020; 7,977,064; 8,610,453; 8,489,163; all expressly incorporated herein by reference. In particular, the new energy-efficient RSFQ logic families (eSFQ and ERSFQ) have zero static power dissipation while retaining all the advantages of conventional RSFQ logic. In these circuits, resistors are replaced with superconducting Josephson junctions performing the role of current limiters. To date, a number of successful eSFQ and ERSFQ integrated circuits have been demonstrated. See, for example, the following articles, all expressly incorporated herein by reference: D. Kirichenko, et al., “Zero static power dissipation biasing of RSFQ circuits”, IEEE Trans. Appl. Supercond., vol. 21, p. 776 (June 2011); M. Volkmann, et al., “Implementation of energy efficient single flux quantum digital circuits with sub-aJ/bit operation”, Supercond. Science & Technology, vol. 26, 015002 (2013); M. Volkmann, et al., “Experimental investigation of energy-efficient digital circuits based on eSFQ logic”, IEEE Trans. Appl. Supercond., vol. 23, 1301505 (June 2013); M. Volkmann, et al., “Operation of practical eSFQ circuits,” Proc. IEEE 14th Int. Supercond. Electronics Conf. (2013).
For many years the prospects of superconducting technology for high-end computing have been stymied by the relatively low capacity of superconducting memories. Very recently, new memory approaches based on magnetic Josephson junctions (MJJs) and on cryogenic magnetic elements have been proposed and are now being extensively studied. See US Patents 2012/0184445; 2012/0302446; 8,270,209; 8,547,732; 2012/0314490; see also WO2013/025994; WO2013/180946; all expressly incorporated herein by reference. In MJJs, critical current can switch between two distinct states corresponding to logical ‘0’ and ‘1’ depending on the magnetization of the ferromagnetic layer(s). Memory circuits using MJJs can be made that are electrically and physically compatible with SFQ circuits. This allows a co-fabrication of memory and digital circuits on the same chip, leading to significant processor-memory architecture advantages relevant to high-end computing. See, e.g., the following papers, expressly incorporated herein by reference: I. Vernik, et al., “Magnetic Josephson junctions with superconducting interlayer for cryogenic memory”, IEEE Trans. Appl. Supercond., vol. 23, 1701208 (2013); T. Larkin, et al., “Ferromagnetic Josephson switching device with high characteristic voltage”, Appl. Physics Letters, vol. 100, 222601 (May 2012); S. Bakurskiy, et al., “Theoretical model of superconducting SIsFS devices”, Appl. Physics Letters, vol. 102, 192603 (May 2013); V. Ryazanov, et al., “Magnetic Josephson junction technology for digital and memory applications”, Physics Procedia, vol. 36, p. 35 (2012); G. Prokopenko, et al., “DC and RF measurements of superconducting-ferromagnetic multiterminal devices”, Proc. IEEE 14th Int. Superconductive Electronics Conf. (2013).
Cooling infrastructure for modern data centers accounts on average for 25-50% of total power. For superconductor systems, the energy efficiency of the entire cryosystem is paramount. The efficiency of available 4 K cryocoolers can reach <400 W/W for higher-capacity units (600-900 W) relevant for high-end computing systems, such as a Linde LR280 cryocooler with 360 W/W efficiency. A future cryogenic supercomputer will have a much smaller footprint than present systems, as the main computing part will occupy a single cryocooler (or perhaps two for redundancy). Besides the cryocooler, the cryosystem energy efficiency depends on minimizing energy losses and heat leaks in the input/output data links and power delivery network. Practical experience with smaller superconductor electronics systems helped the development of a hybrid-temperature hybrid-technology system integration approach to maximize the cryosystem
energy efficiency. The first generation of high-temperature superconductor (HTS) cables for DC bias current delivery were successfully demonstrated to reduce heat leaks in RSFQ electronic cryosystems. See, for example, the following articles, expressly incorporated herein by reference: A. Kadin, et al., “Current leads and optimized thermal packaging for superconducting systems on multistage cryocoolers”, IEEE Trans. Appl. Supercond., vol. 17, p. 975 (2007); R. Webber, et al., “Ultra-low-heat-leak YBCO superconducting leads for cryoelectronic applications”, IEEE Trans. Appl. Supercond., vol. 19, p. 999 (2009); A. Pan, et al., “Development of energy-efficient cryogenic leads with high-temperature superconducting films on ceramic substrates”, Physics Procedia, vol. 36 (2012).
Serious development effort is required in order to take advantage of all of the recent advances capable of addressing the runaway power of high-end computing, and getting superconducting technologies into data centers and supercomputers. A central problem is the relatively low complexity and device density of present superconductor integrated circuits, especially compared to current CMOS technology. The available Nb fabrication processes are generally limited to ˜1 μm linewidth with just a few Nb layers. It is a priority to develop a high-yield, high integration density, planarized fabrication process with linewidth ˜90-250 nm, critical current density JC>10 kA/cm2, and >8-10 Nb wiring layers. Advancing the critical current density to 100 kA/cm2, or using a different junction barrier material than the standard aluminum oxide, is necessary to achieve self-shunted Josephson junctions to eliminate the area-consuming shunting resistors. Another new direction for process development is superconducting-ferromagnetic Josephson junctions (Magnetic Josephson Junctions or MJJs) for magnetic memory and programmable logic. This will enable new programmable functionalities unavailable to superconducting electronics in the past. For example, three-dimensional (3D) integration of processing and memory circuits, fabricated in a single process, should lead to a dramatic gain in the microprocessor performance efficiency, enabling new microarchitectures highly relevant for high-end data-centric computing. All these can be achieved while developing a better understanding of superconductor material issues and actively employing already-developed semiconductor techniques and equipment.
The recent innovations in energy-efficient SFQ digital circuits, eliminating static power dissipation of conventional RSFQ logic, are highly promising. Further reduction of dynamic power dissipation can enlarge the advantage of SFQ circuits over the competition. The next logical step is to implement more functionally significant circuits, such as a microprocessor. One of the common traps with any new technology is the attempt to make better versions of existing solutions which were optimized for older existing technology. The RSFQ-type circuits (eSFQ and ERSFQ) are based on sequential logic, which is different from CMOS combinational logic, implying that the implementation of CMOS-inspired processor microarchitectures and algorithms may not be optimal and will lead to an underutilization of technology potential.
The extremely high clock rate (˜100 GHz) achievable in RSFQ-type circuits fits better to microarchitectures with a high degree of vectorization. In order to keep the processing pipeline full, one should have a fast memory capable of supplying input data and store the results at the same high data rate. This can be alleviated by clever ways of using the internal gate memory. MJJ-based memory circuits integrated in the immediate proximity of processing modules (e.g., as 3D structures) can be an excellent solution. Recent results in MJJ device development provides a path to development of functional fast and energy-efficient memories, including nonvolatile random access memory (RAM) compatible with energy-efficient (e.g., eSFQ) digital circuits. The impact of integrated MJJ RAM and JJ eSFQ processing blocks is difficult to overestimate. This can also lead to the development of programmable digital logic arrays functionally similar to semiconductor field-programmable gate arrays (FPGAs). In addition, the integrated SFQ circuits and non-superconducting magnetic RAM devices are attractive for higher-capacity memories, e.g., main memories.
An energy-efficient, high-bandwidth data interface to room temperature modules and the optical domain is unavoidable in any high-end computing system. There is a need to develop a technology to convert a low-voltage (˜0.3-1.0 mV) electrical digital signal to the optical domain at a high data rate (tens of GHz). This has been a longstanding and extremely difficult problem, which has retarded the integration of ultra-low-power electronics with conventional electronics and fiber-optics. To meet tightly-constrained power budgets, the energy efficiency of data links in exascale systems should be on the order of 2 pJ/bit or less. The degree of amplification at a specific temperature stage can be the guiding principle in the energy-efficiency optimization of data links across different temperature stages available in a cryosystem. HTS multi-bit data cables capable of transmitting low-power signals from 4 K to higher-temperature amplifiers and electro-optical devices (e.g., vertical-cavity surface-emitting lasers, VCSELs) with negligible losses and dispersion will be required. See, for example, O. Mukhanov, et al., “Development of energy-efficient cryogenic-optical data link”, Proc. IEEE 14th Int. Superconductive Electronic Conference (2013), expressly incorporated herein by reference.
It is important to distinguish computers based on RSFQ logic from a set of completely different approaches that are also based on cryogenic Josephson junctions, under the heading “quantum computers”. RSFQ-based computers are digital computers based on classical bits that assume alternate, and not superposed values. In contrast, quantum computers are based on quantum superposition of bits in two or more quantum states, known as qubits. Both analog and digital processors based on superconducting qubits have been proposed, and in some cases developed. These superconducting quantum computers typically require cooling to extremely low temperatures, less than 0.1 K, much colder than the 4 K typical for classical superconducting niobium RSFQ computers. See, for example, the following US patents, expressly incorporated herein by reference: U.S. Pat. Nos. 7,135,701; 7,418,283; 8,284,585; 8,437,168; 8,247,799; 7,605,600; 8,234,103; 7,335,909; 7,889,992; 6,803,599; 6,936,841; 6,838,694; 7,307,275; 6,495,854; 6,649,929; 6,563,310; 6,563,311; 6,459,097; 7,847,615; 7,533,068; 8,283,943; 6,979,835; 6,627,915; 7,253,654. The design and performance of these superconducting quantum computer systems is completely different from the high-performance superconducting classical computers described herein.
The prior art has not yet effectively solved the problems associated with integrating ultrafast superconducting processors with hybrid superconducting/magnetic memories, cryogenic cooling systems, high-speed input/output devices, and room-temperature processors and networks.
With rising energy costs and technical roadblocks, computing system energy efficiency has become the dominating metric dictating the course of future technology development. Superconducting single-flux quantum processors augmented with superconducting-ferromagnetic memory technology can finally break into prominence by addressing the energy efficiency of high-end computing systems. The key innovations just within the last few years have dramatically increased the potential of superconducting electronics, addressing all known critical problems which restricted the use of superconductivity in high-end computing in the past. The present disclosure details several technical advances beyond the prior art, which permit development a cryogenic superconducting computing demonstrator system and ultimately energy-efficient data centers and a new generation of supercomputers.